Papers with generalization gap
Resonance RoPE: Improving Context Length Generalization of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated their potential across a wide spectrum of natural language processing tasks. |
| Approach: | They propose a novel approach to narrow the generalization gap in TSTL scenarios by refining the interpolation of RoPE features for OOD positions. |
| Outcome: | The proposed approach improves performance without additional online computational costs on train-short-test-long scenarios. |
Major Entity Identification: A Generalizable Alternative to Coreference Resolution (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prior work identified annotation differences as one of the main reasons for the limited generalization gap in coreference resolution models. |
| Approach: | They propose an alternative referential task where the target entities are assumed to be specified in the input and the task is limited to the frequent entities. |
| Outcome: | The proposed model generalizes well across domains on multiple datasets with supervised models and LLM-based few-shot prompting. |